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PREFACE 

This  Memorandum  draws  on  three  previous  RAND  studies,  [9],  [’ll], 
and  [12],  to  illustrate  the  use  of  a  probability  prediction  method 
involving  the  logit  model.  It  is  shown  how  the  model  is  used  to  develop 
the  prediction  of  failure  trends  within  aircraft  and  the  prediction  of 
the  reenlistment  rate  of  first-term  airmen.  The  authors  plan  subsequent 
studies  in  which  they  will  use  the  logit  model  to  study  the  interrelated 
maintenance  characteristics  of  different  aircraft  subsystems  and  to 
analyze  the  reenlistment  probabilities  of  Category  IV  airmen. 

To  follow  the  procedures  described  here,  readers  should  be  ac¬ 
quainted  with  regression  analysis  and  contingency  table  methods. 


SUMMARY 


This  Memorandum  discusses  estimation  techniques  of  logit  models 
whose  response  variables  are  not  necessarily  statistically  independent. 
The  statistical  techniques  developed  and  utilized  here  include  the 
Generalized  Least  Squares  approach  of  Zellner  [14];  stepwise  regression 
and  its  relevance  in  estimating  logit  model  parameters;  and  the 
"~ontingency  table"  concept  of  fixed  marginal  probabilities  which  <s 
used  to  develop  a  bivariate  logit  model  that  possesses  certain  sta¬ 
tistical  and  probabilistic  properties  concerning  the  correlation  co¬ 
efficient  and  the  summation  of  individual  cell  probabilities  to  unity. 

The  techniques  are  designed  for  application  to  two  Air  Force 
problems.  The  first  problem  uses  concomitant  information  to  estimate 
the  probability  that  first-term  airmen  reenlist.  The  second  uses  in¬ 
formation  concerning  flying  missions  to  estimate  the  probabilities  that 
interdependent  aircraft  subsystems  fail  during  these  missions. 

Some  applications  to  the  problems  are  presented  in  this  Memorandum; 
however,  this  effort  is  mainly  directed  toward  development  of  the  neces¬ 
sary  techniques.  An  example  illustrating  the  various  techniques  is  given 
in  Sec.  V  of  the  Memorandum. 

In  a  forthcoming  study,  these  logit  models  will  be  applied  to  two 
problems,  analyzing  them  in  detail.  First,  existing  aircraft  sortie 
data  will  be  used  to  study  the  interrelated  maintenance  characteristics 
of  the  different  aircraft  subsystems.  Second,  the  survivability  (as 
defined  in  the  text)  and  reenlistment  probabilities  of  category  IV  airmen 
will  be  analyzed. 
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GLOSSARY 


Endogenous  variable  A  variable  determined  within 

the  framework  of  a  specific 
model. 

Exogenous  variable  A  variable  determined  outside 

of  or  independent  of  the  model. 

AFQT  score  Air  Force  Qualifying  Test  score. 

Opportunity  cost  The  difference  in  dollars  between 

reenlistment  and  the  best  civilian 
alternative  employment,  if  the 
best  civilian  employment  pays 
more  and  the  choice  is  reenlistment. 


I.  INTRODUCTION 


Thi.'  Memorandum  presents  a  probability  prediction  model,  the  logit 
model,  and  its  theoretical  development  tailored  to  be  used  on  two  prob¬ 
lems  explored  in  previous  RAND  studies.  The  first  area  is  aircraft 
performance;  the  second,  airmen  survivability  and  retention.  In  a 
forthcoming  study  we  will  extensively  apply  the  theory  developed  here. 

McGlothlin  and  Donaldson  [12]  describe  an  Air  Force  Base  supply 
officer  who  became  well  known  because  of  his  ability  to  predict  future 
aircraft  performance  based  on  previous  maintenance  records.  The  authors 
report  that  the  5th  Bomb  Wing  at  Travis  Air  Force  Bese  claims  that  its 
method  of  increasing  the  probability  of  sortie  success  depends  on  a 
particular  aircraft  selection  procedure  for  each  sortie.  In  the  same 
vein,  they  discuss  aircraft  labeled  ,,dogs>,,  which  are  inherently  less 
reliable  in  terms  of  sortie  success  than  other  aircraft  of  the  same 
type,  flying  the  same  type  of  sorties.  The  method  in  [12]  and  in  a 
subsequent  study  by  Donaldson  and  Sweetland  [9]  employs  statistical 
analysis  to  discover  the  failure  trends  within  aircraft  and  to  delin¬ 
eate  differences  in  performance  and  maintenance  among  aircraft,  and 
thus  to  identify  the  inherent  dogs. 

In  a  seemingly  different  area,  McCall  and  Wallace  [ll]  have  studied 
the  training  and  retention  of  Air  Force  electronic  specialists.  Approach 
ing  their  analysis  from  an  economic  standpoint,  they  generate  a  supply 
function  for  electronic  specialists.  Their  basic  function  relates  the 
probability  that  first-term  airmen  will  reenlist  to  the  opportunity  cost 
of  doing  so,  where  the  cost  is  defined  as  the  difference  between  poten¬ 
tial  civilian  earnings  and  Air  orce  remuneration.  A  similar  problem 


is  that  of  predicting  the  first-term  enlistment  survivability  of  various 
airmen  groups  ^ 

Although  aircraft  performance  and  airmen  survivability  and  re¬ 
tention  present  different  problems  in  terms  of  subject  matter,  both 
can  be  analyzed  by  probability  prediction  models.  In  the  first  case, 
the  probability  of  failure  is  considered  an  endogenous  variable,  de¬ 
pending  upon  one  or  more  exogenous  variables  such  as  type  of  sortie, 
previous  maintenance  actions  on  the  total  aircraft  and/or  itB  subsystems, 
and  length  of  maintenance  service.  In  the  second  case,  the  probability 
of  reenlisting  is  an  endogenous  variable,  depending  on  the  opportunity 
cost  of  reenllsting;  the  probability  of  survival  is  an  endogenous 
variable  depending  on  exogenous  variables  such  as  age,  prior  education, 
race,  and  initial  AFQT  score.  These  two  problems  use  similar  pre¬ 
dictive  models  and  statistical  tests. 

To  treat  such  problems  and  to  indicate  answers  to  others  falling 
within  the  same  framework,  we  analyze  a  predictive  model  known  as  the 
logit  model.  In  this  model,  the  probability  P  that  an  event  E  will 
occur  depends  on  a  vector  X'  ■  (X^,  ...,  Xr)  of  exogenous  variables  and 
is  written 


where  j}'  ■  (0^ . Pn)  is  a  vector  of  the  regression  coefficients  to 

be  estimated.  This  is  known  as  the  logistic  function.  The  logit  of 
P(E) ,  then,  is  simply  the  following  transformation: 

^Survivability  denotes  the  capability  of  an  airman  to  complete  his 
first  term  of  reenlistment. 
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(lb) 


logit  [P(E)  ]  -  ln[1  r^p(iy1  “  £'*  * 


The  occurrence  (or  nonoccurrence)  of  E  is  usually  assumed  to  be 
independent  of  the  occurrence  (or  nonoccurrence)  of  other  events.  It 
is  well  known,  however,  that  aircraft  subsystems  do  not  fail  independently 
of  one  another  during  sorties.  To  verify  this  statement  we  have  observed 
the  writeup  histories  of  three  aircraft  subsystems  for  200  sorties: 
radar,  inertial  navigation,  and  fire  control.  Table  1  shows  the  ob¬ 
served  cell  frequencies  and  the  expected  cell  frequencies  under  the  hy¬ 
pothesis  of  the  mutual  independence  of  these  writeups. 


Table  1 

FREQUENCIES  OF  DIFFERENT  EVENTS  FOR  THREE  SUBSYSTEMS 


Frequency 

SSS 

SSF 

SFS 

FSS 

SFF 

FSF 

|ffs  | 

FFF 

Total 

Observed 

86.0 

10.0 

18.0 

9.0 

m 

S9 

200.0 

Expected 

72.3 

19.8 

23.5 

12.7 

U)C! 

ED 

in 

200.0 

NOTE:  S  stands  for  no  writeup  and  F  for  writeup.  Thus,  the 
event  SFS  is  no  writeup  of  the  radar,  writeup  of  the  inertial 
navigation,  and  no  writeup  for  fire  control. 


The  analysis  is  essentially  that  of  a  2x2x2  contingency  table  in 

which  there  are  two  outcomes  for  each  subsystem:  S  for  no  writeup  and 

F  for  writeup.  Let  f  ^  be  the  observed  number  of  sorties  in  the  cell 

(i,j,k),  where  i,  j,  k  "S,  F.  Similarly,  let  f  .  be  the  expected 

i  jk 

number  of  sorties  in  the  same  cell.  Under  the  assumption  of  mutual  in¬ 
dependence  of  the  writeups,  the  expected  fraction  p  ,  (of  the  total 

1  JK 

number  of  sorties)  in  the  cell  (i,j,k)  is  equal  to  the  product  of  the 


marginals;  that  is  p 


i 


ijv  *  pi..p.j.p..k’  "hete  pi..  ’ 
S,  F,  and  similarly  for  the  other  two  subsystems. 

We  then  compute  the  statistic 


j,k 


Then  f 


tjk 


Np 


ijk* 


x2  -  £  ^  ^  • 

l,j,k  'ljk 

The  associated  chi-square  value  is  49.0,  with  a  single  degree  of  freedom. 
This  value  is  significant  beyond  the  0.0001  level.  Thus,  the  hypothesis 
of  mutual  Independence  of  writeups  for  these  three  subsystems  is  emphat¬ 
ically  rejected.  This  dependence  of  aircraft  subsystems  as  well  as  the 
dependence  of  airmen  subgroups^  has  led  us  to  couch  our  specific  problems 
in  a  dependent  equation  model. 

Section  II  presents  the  basic  theory  of  the  logit  model.  First, 
we  briefly  review  the  literature  on  the  use  of  these  models,  and  then 
describe  the  model  as  it  applies  to  our  needs.  This  includes  a  theo¬ 
retical  explanation  of  the  Generalized  Least  Squares  approach  that 
Zellner  [14"1  uses  to  estimate  "seemingly  unrelated  regression  equations." 
A  short  discourse  on  stepwise  regression  and  its  relevance  in  estimating 
the  parameters  of  logit  models  is  also  included.  Section  III  describes 
the  estimation  procedures  that  are  subsequently  used  to  establish  the 
posited  relationships.  In  Sec.  IV,  we  present  a  bivariate  logit  model, 
describe  its  mathematics,  and  give  reasons  why  a  generalized  dependent 
logit  model  might  be  better  than  individual  independent  logit  models. 
Section  V  illustrates  a  problem  solved  by  the  methods  described  here. 

And  the  Appendix  contains  derivations  of  some  equations  in  the  text. 


Airmen  subgroups  are  any  groups  that  can  be  distinguished  on  some 
predetermined  basis:  for  example,  Caucasian  and  non-Caucasian;  high 
school  graduates  and  high  school  dropouts.  These  groups  are  clearly 
neither  independent  nor  homogeneous. 
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II.  BASIC  THEORY 

Much  has  been  written  about  the  theory  and  application  of  the 
logistic  function.  One  pioneer  in  the  use  of  the  logistic  function 
is  Joseph  Berkson  [2-6,7].  He  has  applied  the  logistic  function  to 
quantal  data  (death  or  survival)  in  bioassay  testing  of  animals;  that 
is,  the  testing  of  a  chemical  compound  in  animals  to  determine  lethal 
doses  for  various  percentages  of  t lie  particular  animal  population. 

Other  important  papers  concerning  the  logistic  function  have  been 
written  by  Cox  [8],  Dyke  and  Patterson  [10],  and  Walker  and  Duncan  [13]. 

Suppose  that  each  observation  in  a  series  of  trials  can  take  one 
of  two  forms,  which  in  our  particular  applications  are  "reenlistment" 
or  "failure  to  reenlist"  for  the  airmen  data,  and  "subsystem  writeup" 
or  "no  subsystem  writeup"  for  the  aircraft  maintenance  data.  Suppose 
that  corresponding  to  each  trial  there  are  one  or  more  independent 
variables  upon  which  the  outcome  of  a  particular  trial  is  suspected  to 
depend.  Since  each  observation  (the  dependent  variable)  is  dichotomous, 
we  denote  the  observations  by  0  and  1  and  thus  obtain  a  sequence  of  0's 
and  l's  for  the  outcome  of  the  trials. 

To  formulate  the  logistic  function,  let  Y^,  ...,  Y^  be  mutually 
independent,  binomially  distributed  random  variables  and  X'  ...,  X1  be 
a  set  of  fixed  s-vectors;  that  is  ,  X^  =  (XiQ,  X^ ,  . . . ,  Xi  g  j) .  We 
assume  s  <  n.  Further,  let  p^  be  the  probability  of  success  and  be 
the  number  of  trials  at  the  ith  stage,  i  =  1,  2,  ...,  n.  Then  the 
binomial  distribution  may  be  written: 


(2) 


bi(Yi’  pi’  Ki) 


where  =  0,  L,  K^,  ,  i  =  1,  2,  n.  It  is  suspected  that  is 

related  to  the  vector  X^. 

One  might  conjecture  a  linear  relationship  between  p^  and  X^  , 
i  =  1 ,  . . . ,  n.  That  is  , 


Pi  ^o40ixii  + 


, .  +  B  ,X  ,  . 
s-1  i ,s-l 


However,  p^  must  lie  between  zero  and  unity  so  that  the  procedure  for 
estimating  the  parameters  Bg>  Bp  •••»  Bg  ^  would  be  constrained  by 
this  condition  on  p^  ,  i  =  1,  n.  There  are  functional  relationships 

between  p^  and  X  ,  i  *  1,  ...,  n  that  are  free  of  the  above  constraint 
but  yet  insure  the  restriction  that  lie  between  zero  and  unity  is 
not  violated.  One  such  relationship  is  the  logit  model  given  by 

Pi 

(3)  logit  (Pl)  -  In  y~  p  "  "  I'Xi  , 

where  j)'  ■  (Bq,  Bg  Is  the  regression  vector  upon  which  we  draw 

inferences.  Rewriting  (3),  we  see  that 


<4)  pi - L^FxT  • 

1  +  e 

Ordinarily,  X^q  =  1  (i  ■  1 . s-1)  so  that  Bg  is  the  intercept 

parameter.  We  shall  abide  by  this  convention. 

We  place  emphasis  on  the  logit  model  in  this  Memorandum  for  the 
following  reasons: 

1.  The  procedure  for  estimating  £  is  free  of  constraints  such 
as  those  that  exist  for  the  linear  model. 

2.  Estimates  of  the  logit  model  parameters  are  easily  obtained 
and  possess  desirable  statistical  properties. 


3.  In  many  applications,  including  those  discussed  here,  the 
model  has  empirical  Justification. 


4.  Much  ha6  been  written  that  we  can  use  concerning  the  logit 
model  and  its  application. 

There  are  two  standard  procedures  for  estimating  the  regression 

coefficient  vector  (0_,  0.  »  ,  0  .)--maximum  likelihood  and  minimum 

vJ  JL  S  -  1 

chi-square.  Berkson  compares  these  methods  in  ["6]. 

We  solve  the  equations^ 


(5a) 


52  K  (Y  /K  -  p.)  -  0 
i-1 


and 


(5b) 


E  •  ?i>  ■ 0 


i-l 


A  A 


j  —  o,  . .  .  i  s  -  1|  for  £  —  (0q  »  0  ^  >  •  •  •  ^s-1^’ 


(6) 


1  +  e 


-(£'Xi) 


£  is  then  the  maximum  likelihood  estimate  of  J3. 
The  solution  of  the  equations^ 


(7a) 


A  [Pjd  -  Yi/K1)  +  (1  -  p1)Yi/Kl] 


^  K 

i-l 


Pid  ■  V 


(Yi/Ki  -  »i>  "  0 


Equations  (5a)  and  (5b)  are  the  normajj  equations  whose  solution 
yields  the  maximum  likelihood  estimates  of  £.  The  derivation  of  these 
equations  is  sketched  in  the  Appendix. 

^Equations  (7a)  and  (7b)  are  the  normal  equations  whose  solution 
yields  the  minimum  estimates  of  1).  The  derivation  of  these  equations 
i6  also  sketched  in  the  Appendix. 
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and 


(7b) 


Vvv  [?,<!  -Y*t)  -mi  -PiVV 

ImU 


ij 


Tl(i  -  Pi> 


(Yi/Kl 


Pi) 


j*l .  8-l,in  terms  of  .8,  where 


(8) 


1 


1  -d%)  ’ 

(1  +  e  ) 


yields  the  minimum  chi-square  estimate  Jf  of  j).  The  solution  of  each 
set  of  equations  (5a,  5b)  and  (7a,  7b)  usually  requires  an  iterative 
approach.  This  is  so  because  the  coefficients  are  functions  of  p  (or  p) , 
and  p  (or  ^>)  is  not  linear  in  the  parameters. 

Berkson  [6]  discusses  the  properties  of  |[  and  £  and,  to  some  extent, 
their  relative  merits.  For  example,  the  estimates  are  best  asympto¬ 
tically  normal  (BAN)  and  hence  they  have  the  same  asymptotic  properties 

*  (V  A 

as  £.  That  is,  both  j}  and  g  are  consistent,  asymptotically  efficient, 
and  asymptotically  normally  distributed  tstimates  of  j3.  In  the  com¬ 
parative  sample  Berkson  worked  with  he  demonstrated  that  the  mean  square 
errors  of  the  minimum  chi-square  estimates  were  smaller  than  the  mean 

square  errors  of  the  corresponding  maximum  likelihood  estimates.  He  also 

2 

discusses  the  minimum  "logit  x  »'  which  has  the  desirable  property  that 

the  parameters  ji  can  be  estimated  simply  and  directly. 

2 

The  "minimum  logit  y  estimate"  of  £  is  the  solution  to  the 

2 

following  quantity  defined  as  the  "logit  y 


logit  y2  »  £  P1  ( l  -  pJU,  -  V2, 
i-1 


(9) 
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A 

where  l =  In  / ( 1  -  p^)  is  the  observed  logit,  and  l ^  =  £'X  is  the 

estimated  logit.  The  normal  equations  for  obtaining  the  minimum  logit 

2  t 

X  estimate  of  £  can  easily  be  written: 

N 

(10a)  S  "^(i  -  p±) (^±  -  2£)  -  o, 

N 

(iob)  2  nipi(i  ■  pi>xi j(£i  ■  v  *  °- 

Unfortunately,  the  usual  logit  model  as  it  is  structured  applies 
only  to  binomially  distributed  variables  and  thus  cannot  be  applied 
directly  to  a  multinomial  situation.  Because  of  the  dependent  nature 
of  cur  data  (in  the  sense  that  aircraft  subsystem  writeups  are  cor¬ 
related)  and  because  efficient  linear  regression  procedures  are  avail¬ 
able  when  the  observed  proportions  r^  -  X^/Kj  are  not  merely  zero  or 
unity,  we  now  discuss  these  applicable  procedures.  The  substance  of 
the  following  derivations  can  be  found  in  Zellner  [14]  and  Zellner  and 
Lee  [15]. 

Suppose  that  independent  samples  of  size  n^,  n^ . n^  are 

observed  on  items  (such  as  airmen  reenlistments,  and  aircraft  sub¬ 
system  writeups)  that  have  a  dichotomous  response.  Suppose  that  r^ 
is  the  proportion  of  "l's"  for  the  i^  sample.  We  then  assume  that 

(11)  ri  "  Pi  + 

where  u^  has  a  binomial  distribution  with  mean  zero  and  variance 
[p1(l  -  pi)]/n1,  and  where  “Pi  <  ui  <  1  “  Pt* 

^The  derivation  of  these  normal  equations  is  also  sketched  in 
the  Appendix. 


From  (11)  we  see  that 


(12) 


h  Pi  /  1  +  "I'h  \ 
1  -  ri "  i  -  Pi  v  -  ui/(1  -  pi V 


Thus,  letting  Z  -  In  - —  and  using  (3)  we  have 

i  ri 


(13) 


zt  *  +  In  (1  +  u^p^  -  In  [1  -  ui/(l  -  p1)]. 


Expanding  the  last  two  terms  of  (13),  we  obtain 


(14a) 


*t-l'»i+pi(i-Pi)+V 


-1 


where  denotes  the  remainder.  is  of  order  n^  in  probability, 

and  its  omission  will  not  affect  the  asymptotic  properties  of  the 


estimates  of  Z^. 
Thus 


(14b) 

where 


B  tl J^/p ^(  1  ■- p ^)  ,  1  “  1»  m* 


Hence,  the  least  squares  of  estimate  £  of  £  is  given  by 


(15) 


1*  -(x'£  ^ 


where^  is  a  diagonal  matrix  whose  i^  element  on  the  diagonal  is 


,-l 


[n  i  Pi  ( 1  -  P^)]"  •  The  least -squa  res  procedure  to  obtain  jj  is 


an 
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application  of  Aitken's  generalized  least  squares  [l],  The  variance- 

* 

covariance  matrix  of  £  Is  given  by 

(16)  Var  ji*  «^X'£  +  o(n‘1)  , 

where  o(n^)  signifies  a  matrix  whose  elements  are  of  smaller  order 
than  n^  in  probability. 

As  demonstrated  in  the  Introduction,  with  respect  to  subsystem 
writeups,  while  there  is  no  apparent  intracorrelation  among  writeups 
within  a  subsystem,  there  may  be  a  correlation  among  groups  of  sub¬ 
systems  with  respect  to  writeups.  Thus,  we  make  some  changes  in  the 
model  (14)  for  the  case  of  i  characteristics.  We  again  follow  Zellner 
and  Lee's  lead.  Let 


Z," 

m  « 

JO  ...  0  ^  . . .  Rg 

~e  ' 

V, 

-1 

01 

-1 

z„ 

0  J  ...  OR.  ...  R 

• 

v„ 

—2 

-  - 1  -s 

• 

a 

-2 

(17) 

fi 

m 

p0 1 

„L. 

•  a 

•  a 

a  • 

h 

“T 

a 

hm 

0  0  JR.  R 

-  - 1  — s 

K 

—  m 

1 - 

l< 

where  J  is  an  n-vector,  each  of  whose  components  is  unity: 


— 'm  "  (Xml'  •••'  X«n)'  '  1  •  2  ‘  ••••  s> 

V1J  ’  Ulj/Pij(l  ■  PiJ>- 

For  the  remainder  of  Sec.  II,  the  X^  vectors  will  denote  the  vector 
of  trials  for  the  i^  exogenous  variable  (the  vectors  in  the 
footnote) . 

We  require  that  the  Z^'s  each  have  the  same  dimension--N  x  1. 
Thus,  writing  (17)  in  compact  notation  we  see  that 


(18) 

where 


Z  »  X  |  +  V  , 


Z'  “  (Z^  i  Z_2 . —  * 

fj,  *  (  Bq^  ,  ^0£  ’  *  *  *  *  ®  i  ’  *  *  *  *  » 

V’  -  (V[,  ....  Vj) 


It  Is  important  to  note  the  relationship  between  the  X  vectors  at 
the  bottom  of  page  5  and  the  R  vectors.  The  individual  components 
of  the  vectors  are  such  that  XX'  -  R’R,  where  we  define  the  matrices 
X  and  R  as  follows; 


X  -  [X.  . ..  X  ] 
—1  — n 


and 

R  -  [Rl  ... 


where  the  Xj»  represents  the  val 
the  ifc^  trial. 


X11X21  "•  Xnl 


X, 

X, 

X 

L  18 

2s 

ns 

Xll 

• 

• 

• 

• 

CM 
•— < 

x. 

Is 

• 

x  , 

X  .  ... 

X 

.  nl 

n2 

ns 

of  the  )  exogenous  variable  for 
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and 


(19) 


I  0  ...  OXj 

ox! 


0  0 


i  *1 


x  J 

— s  -* 


The  model  for  determining  the  proportion  of  airmen  reenlistments 
is  the  generalization  of  (14)  that  Zellner  and  Lee  ["151  discuss.  It 
can  also  be  written  as  (18)  with  the  same  definition  for  Z1  and  V1, 
but  with  *  (£j,  .  .  .  ,  Bp  ,  and 


(20) 


X !  0  ...  0 
0  2^2  ...  0 


0  0  ...  X. 

—I 


where  denotes  the  set  of  exogenous  variables  for  the  i^1  distinct 
group  of  airmen. 


The  variance -covariance  matrix  for  V,  which  is  independent  of  Xf , 
will  now  be  developed  in  general.  Again,  following  the  development  of 
Zellner  and  Lee  [l5l,  the  variance -covariance  matrix  T  is  given  by 


(21) 


and 


E11  *12  •**  El£ 

nr  y 

21  22  **•  21 


Zll  E£2  **•  Tll 


V 


(11,  .... 


1). 


pjj'(v)  is  the  probability  that  an  individual  in  the  group  has 
value  "1"  in  both  characteristics  i  and  j.  Thus, 


(24) 


for  i  i  j. 


Since  we  want  to  test  hypotheses  about  £  and  derive  corresponding 
confidence  intervals,  we  now  discuss  some  general  principles  con¬ 
cerning  testing  of  hypotheses  and  derivation  of  confidence  regions  for 

the  estimated  parameters.  Suppose  we  wish  to  test  the  hypothesis  that 

★ 

the  s-dimens ional  parameter  space  j3,  based  on  the  estimate  £  given  by 
(15),  can  be  reduced  to  a  s 1  <  s  dimensional  space.  The  null  hypothesis 
Hg  can  be  written 

(25)  Hq:  C  £  =  0  , 

where  C,  exhibits  the  restriction  under  H...  To  test  Hn, 

(s-6  )x6  0  0 

Zellner  [14]  gives  as  a  test  statistic: 


_  (n-s)  z'r'1x(x'r"1x)'1c'[c(x,E'1x)'1c']c(x,r'1x)'1x'r'1z 

(Zb)  r  ■  m  f  *i .  ■  ■  —  -  .  - ■ 

s-6  ,n-s  (s-s')  .  -i  .  -1  .  -1  -1  .  -1 

Z'E  Z-Z'E  X(X'E  X)  X'E  Z 


That  is,  we  reject  Hn  at  the  Q'-level  of  significance  if  and  only  if 


F  ,  >  F  , 

s-s  , n-s  s-s  ,n-s  ,a 


F  ,  is  the  upper  a  percentile  of  the  F  distribution  with  s-s1 

S  *8  jll“8  , O' 

and  n-s'  degrees  of  freedom. 
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To  obtain  a  confidence  region  for  we  note  that  since  |  is  a  best 
linear  unbiased  estimator  of  £,  then  based  on  the  normality  assumption 
of  Z,f 


(28) 


s  ,n-s 


/ n-s\  (£*  -  £)'  (B*  -  B) _ 

\  8  )[z'r'lz-z'T~]■x(x'T.~]■x)~1x.'r~1z^ 


Thus,  a  ( 1  -o)  level  of  confidence  region  for  j3  is  given  by 


(29) 


/n-s\  (I*  -  j)  '  (£*  -  £) _ 

\  8  /  z,^‘1z-z'7:'1x(x'r'1x)'1x'r‘1z 


where  F  is  similar  to  F  , 

s,n-s;&  s-s  ,n-s;» 

In  applying  the  logit  models  considered  here,  we  note  that  there 
is  a  relationship  between  the  dependent  variables  and  several  exogenous 
or  independent  variables.  For  example,  the  failure  (or  success)  of  inter¬ 
dependent  aircraft  subsystems  is  related  to  exogenous  variables  such  as 
type  of  mission  (for  example,  escort  mission,  bombing  mission),  length 
of  mission,  maintenance  actions  prior  to  the  mission,  and  time  of  previous 
overhaul.  In  these  applications  it  is  important  to  identify,  if  possible, 
those  exogenous  variables  which  "best  explain"  the  behavior  of  the  de¬ 
pendent  variables  in  the  sense  that  by  eliminating  those  exogenous 
variables  that  are  not  "best,"  little  is  lost  in  the  accuracy  of  pre¬ 
dicting  the  outcome  of  the  dependent  variables.  Stepwise  regression 
affords  a  methodology  for  choosing  a  subset  from  the  set  of  exogenous 
variables  that  are  among  the  best  with  regard  to  "explaining"  the 

^The  asymptotic  normality  of  Z  is  a  consequence  of  the  asymptotic 
normality  of  u.  The  rate  of  convergence  of  u  to  normality  depends  on  £. 
For  pi  small  or  large,  the  rate  of  convergence  of  u^^  is  slower  than  for 
moderate  values  of  p^. 
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behavior  of  the  dependent  variables.  Briefly,  the  stepwise  regression 
"explains"  the  behavior  of  the  dependent  variable  by  maximizing  the 
regression  sum  of  squares  sequentially  until  some  prespecified  criterion 
is  achieved.  That  is,  we  examine  all  simple  regressions^ 


(30) 


Z\  -  0  X*.  +  e. 
i  v  vi  i 


X' 

vi 


(x  -  x  ); 

vi  V 


zi 


( Zi  -  2);  i  -  1,  2,  ...,  n, 


1,  2,  ...,  s. 


We  then  choose  0 ' 
v, 


such 


that 


Next,  we  let  Z"  ■  Z\  -  0  X  ,  and  examine  the  s-1  remaining  re- 
i  i  Vj  v^  B 

gressions.  Thus,  we  consider 


(32)  Z’^  -  Pv.X^,t  +  €  1 , 

Xv '  i  *  (xv*i  ”  xv ' )  •  1  “  ^ »  2,  ....  n, 

Now  we  choose  P  such  that 


1,  ...',  S,  V*  ^  v ,  • 

L 


^We  assume  E(ei£i)  ■  6 

6U  -  1,  6n  -  0,  i  4j. 

tt 

Regression  s.s.(0^), 
due  to  0^. 


where  ia  the  Kronecker  delta,  i.e., 
1 ,  2  is  the  regression  sum  of  squares 
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This  procedure  is  continued  until  one  of  the  following  occurs:  ( 1)  a 

predetermined  amount  of  total  variation  is  explained  by  the  subset 

(0  ,  g  );  (2)  a  predetermined  number  of  regression  coefficients  » 

V1  \  ' 

say  Sp  is  chosen;  or  (3)  some  other  practical  cutoff  point  is  reached. 

As  indicated  in  the  footnote  on  page  15  ,  this  development  assumes 
uncorrelated  and  homoscedas tic  observations;  however,  the  modifications 
discussed  earlier  enable  us  to  apply  the  stepwise  procedure  to  our 
problem. 
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III.  ESTIMATION  PROCEDURES 

In  describing  the  techniques  used  to  estimate  the  parameters  of 
the  logit  model,  we  divide  the  following  discussion  into  two  major 
parts:  (1)  the  estimation  of  single  equation  models,  and  (2)  the 

estimation  of  dependent  equation  models. 

A  single  equation  model  is  one  that  estimates  the  parameters  of 
a  single  posited  relationship  when  this  relationship  is  uncorrelated 
with  any  others.  For  example,  the  equation  that  relates  the  proba¬ 
bility  of  reenlistment  for  airmen  to  exogenous  factors  falls  into  this 
category.  Consider  now 

(34a)  Zt  *  £'3^  +  i  -  1 . . 

where 

~  (30»  •  •  •  *  0S _ i) 

-i  =  (x0i’  xii’  •••*  xs-i,i> 

ei  =  ui(ri(1  -  ri))"1 

ECc^  =  0, 

and  for  i  -  J,  E(eiej)  =  0^(1  -  r^f1 

Using  the  procedure  of  ordinary  least  squares  (OLS)  ,  we  can  obtain 
* 

an  estimator  b^  of  j3: 

b*  =■  (X'X)-1  X’Z, 

where 

X  =  (X-,  xj,  ...  x;). 

* 

The  estimator  b  is  unbiased  and  is  equal  to  the  maximum  likelihood 

M 

estimator  l)  (assuming,  of  course,  that  the  are  normally  distributed). 
The  disturbances,  e^,  exhibit  heteroscedasticity ,  however,  and  hence 
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b  is  not  a  best  linear  unbiased  estimator  (BLUE).  To  get  an 

asymptotically  BLUE  estimator,  we  use  the  following  procedure.  Since 

* 

(34)  is  a  single  equation,  it  is  easy  to  obtain  .  First  we  cal¬ 
culate  the  weights 


(34b) 


if- 

wi  *  tniri(1  “  ri^  >  1  “  •••»  n» 


where  r^  is  the  observed  proportion  of  "l's"  for  the  i  sample  as 
given  in  (11).  A  weighted  regression  is  then  performed  using  OLS 
on  the  following  equation: 

(35)  wizi  =■  J3 'w12ii  +  ^71- 

* 

The  OLS  estimator  j}  of  £  is  now  an  asymptotically  BLUE  estimator  and 

2 

is  equal  to  both  the  MLE  £  and  the  minimum  logit  y  estimator  of  (14). 

■k  * 

In  addition,  is  exactly  the  same  as  the  j}  of  (15). 

However,  the  simple  procedure  does  not  suffice  for  dependent 
equation  models  ,  that  is  those  that  have  two  or  more  posited  re¬ 
lationships  which  are  interdependent.  Two  types  of  these  models  were 
explained  in  (17)  and  (20).  Assuming  that  the  entire  model  contains 
l  equations,  we  can  write  the  system  as 


(36a) 


h  XL  0  •••  0  uL 

0  X  ...  0  £ 


•  h\  \h  ±1 
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or 

(36b)  Z  =  Xj3  +  u  . 

As  noted  above,  the  XQ  are  not  necessarily  mutually  exclusive 
(see  (17)),  nor  is  it  essential  that  all  the  &Q  be  different.  The 
solution  of  (36)  can  be  written  as 


(37) 


l  =  (X'rtr^'rt  . 


The  method  that  Zellner  has  devised  [l4l  is  used 


,-l 


to  estimate  the 


matrix  E  .  Essentially,  it  estimates  each  of  the  i  individual 
weighted  regressions  by  OLS.  Thus 


(38) 


4  =  <W‘i£^9.  9-1 . 1. 

* 


The  estimator  bQ  is  used  to  obtain  the  MLE  of  the  disturbance  vector 

A 

u0  for  equation  0: 


(39) 


=  -e  "  »  Q  =  i,  —  ,  £ . 


We  can  then  compute  the  matrix 


(40) 


-1 


11  •**  nu 


-i-l 


°n  ■■■  °u  J 


SUCK  that  z:u  la  of  dimension  f  x  f  and  has  the  general  dement  a 


given  by 


00 


700  =  n  CH£H0)-  „  S  30iG0i  , 


9,0  =  1 


>  •  •  •  | 


l. 
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1  _1  .  i  l 

The  matrix  E^  is  an  estimate  of  E  .  Replacing  E  by  E^  in  (37) 

"fc  'fc  'ft 

yields  estimates  £  for  £,  where  £  '  =  (j3^  1  ,  . . .  ,  £q'  , 

* 

These  estimators  consider  explicitly  the  intercorrelation  between 


the  l  equations. 

The  principal  advantage  of  using  the  above  method  is  that  the 

"fc  41  ■jlf 

estimators  are  more  efficient  than  the  bg.  This  is  especially 
helpful  to  the  decisionmaker  when  various  constraints  are  imposed  on 
the  estimators  as  in  (42)  below. 

However,  there  is  a  gain  in  efficiency  using  Zellner's  method  only 
if  correlation  exists  between  the  dependent  variables;  that  is,  only 
if  EtZ^)  t  0,  for  all  0,  0  such  that  0^0. 

If  there  is  a  good  reason  to  believe  that  E(Z^Zg)  =  0,  then  there 
is  a  considerably  simpler  method  that  may  be  used  to  estimate  (17). 

Thus  ,  let  ^  —  (Z^  ,  . . .  ,  Z'q)  ,  X  =  (X^  ,  •  • .  ,  Xg.^)  >  anc^  &  •••»  1 

Then  the  following  regression  may  be  estimated  by  OLS: 

s+f  l 

(4i)  i  ‘  +  £  efixfi  +  u  , 

0-1+1  6  9  0=1  9  6 


where  XQ  =  0,  1  for  all  0  e  [l,£].  Similarly,  the  parameters  of  (36) 
can  be  estimated  by  applying  OLS  separately  to  each  equation  of  the  model. 

The  system  (17)  we  wish  to  estimate  is  merely  the  general  system 
(36)  with  linear  restriction  on  the  estimates;  viz.,  the  "slope"  co¬ 
efficients  are  constrained  to  be  the  same  among  the  equations,  while 


That  is, 


Var(£*) 

Var(b^) 


for  all  0. 
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the  intercepts  are  allowed  to  vary.  To  handle  linear  restrictions  in 
the  present  problem,  we  must  minimize  the  following  equation  with 
respect  to 


(42) 


(Z  -  Xfi) '(Z  -  X£)  -  \(R£  -  r)  , 


where  Z,  X,  and  are  defined  in  (18)  ,  X  is  a  row  vector  of  Lagrange 
multipliers,  and  R  and  r  are  a  matrix  and  a  column  vector,  respectively, 
which  together  define  the  restrictions  on  the  system.  The  estimator, 
will  satisfy  the  relationship 


> 


where  R  is  a  matrix  with  a  row  for  each  restriction.  For  example,  the 
restriction  matrix  for  (17)  would  be 


r 

au  *•*  an  **•  a it 

CP 

o 
>— * 

• 

• 

• 

• 

•  • 

• 

•  • 

• 

Box 

• 

• 

« 

Bi 

• 

3  4  .  •  •  •  A  «  *  •••  A  4  _ 

£1  H  It 

• 

B 

8 

where 

(i) 

fliJ 

-  o, 

for 

i,  j  *  1  ...  1 

(ID 

aU 

-  1, 

for 

)  -  l  +  i;  i  - 

and 

T  -  l.s 

(iii) 

“ij 

-  1, 

for 

j  “  t  +  8  +  i. 

Minimizing  (42),  we  obtain  the  following  expression  for 
(43a)  §.A  “  |  +  (X'X)'1R'[R(X’X)'1R']"1(r  -  R|)  , 
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where  3  ■  (X'X)  ^X'Z^  the  unconstrained  least-squares  estimator.  The 
covariance  matrix  of  the  estimator  £A  is  given  by  V(£A) : 

(43b)  x  v(£A)  <■  v(|)  -  vc^rTrCx'xj^r'I^RVcI). 

To  account  for  the  correlated  error  term  between  equations,  we  use  the 
following  estimator  of  0: 

(44a)  bu  «  b  +  (X'E“1X)‘1R,[R(X,^1x)"1R,f  *(£  -  R£)  , 

and  the  covariance  matrix  is 

% 

(44b)  V(bA)  -  V(b)  -  V(£)R'[R(X,^1X)'1R']"1RV(b). 
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IV.  BIVARIATE  LOGIT  MODELS 

The  development  to  this  point  has  been  oriented  toward  univariate 
logit  models.  That  is,  we  have  considered  the  probability  that  an 
aircraft  system  would  either  "fail"  or  "not-fail,"  that  an  airman  would 
either  "reenlist"  or  "not-reenlist ,"  or  that  the  subject  under  con¬ 
sideration  would  possess  some  characteristic  on  a  "yes-no"  basis. 

Then  we  analyzed  the  interdependent  relationships  of  the  various  sub¬ 
jects.  However,  situations  arJ je  in  which  a  subject  can  possess  more 
than  one  characteristic,  each  on  a  "yes-no,"  or  "pass-fail"  basis. 

For  example,  consider  any  weapon  system  with  more  than  one  com¬ 
ponent  part  (or  in  some  circles,  "black  box"!.  In  these  instances  it 
would  be  preferable  to  incorporate  the  dependence  of  the  component 
parts  directly  into  the  probability  functions.  Wt  have  done  this  below 
for  the  simplest  case--that  of  two  variab les --such  that  the  marginal 
distribution  of  the  probability  of  "success"  for  each  variable  is 
represented  by  a  logistic  function. 

Consider  the  following  2x2  contingency  table  for  two  characteristics, 
say  7}^  and-?^,  of  some  subject  on  a  "success  -  fai  1"  basis  such  that 
success  is  represented  by  a  one  and  failure  by  a  zero, 
where 


Table  2 

CONTINGENCY  TABLE  FOR  CHARACTERISTICS  ^  AND  7/ 


^1 

Success 

Fail 

Marginals 

Success 

P11 

P10 

Pl. 

Fail 

poi 

P0Q 

po. 

Margina Is 

P.l 

p.o 

P  _ 
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We  now  take  into  acconnt  the  intercorrelation  and^*  Let 

p  (i),  v;here  r,  6  =  0,1  is  the  probability  that  on  the  i*"^  trial  the 
characteristic^  takes  on  the  value  r,  and  ^  the  value  s. 

If  7}  and  7^  are  independent  characteristics  we  can  then  write 

r  vu  v2i  i  -i 

(45a)  Pi  (i)p =  [(1  +  e  )(l+e  )J  Hpn(i), 

where 

Yu  "  -(t>0  +  ttlXil  +  +  VlXi,s-l>’ 

and 

Y2i  =  +  PlZil  +  “■  +  Pt-l*Zi’t-l)> 

However,  (45a)  does  not  hold  if  characteristics)^  and  are  dependent. 
Regardless  of  whether  or  not>/^  and 7^  are  independent,  the  relation¬ 
ships  of  Table  2  must  hold.  Namely,  p^ti)  +  p^i)  *  Pi  (*■)»  Pu(0 

+  P0l(i)  «  P  ^i).  P10<i)  +  P00(i)  “  P.0(l)»  3nd  p01(1)  +  P00(1)  "  P0.(i)- 

We  shall  require  that  these  marginal  probabilities  be  logit  models  so 
that  when  we  deal  with  these  characteristics  marginally  (separately) 
we  can  use  the  theory  already  developed. 

Thus  , 

f  Yu 

(46)  (i)  pi  (1)  =  j^l  +  e 

T  V2i 

(ii)  P  x(i)  -  [l  +  e 

A  way  to  write  p^(i)  as  a  bivariate  logit  model  to  include  the  situa¬ 
tion  for  which'*/  and7/9  are  dependent  while  preserving  the  requirement 
that  (46)  holds,  is  to  write 
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(45b) 

so  that  P10(i),  Pqi  (i)  >  and  Pqo^)  are  chosen  in  such  a  way  that  the 
relationships  of  Table  2  hold  as  well  as  Eq.  (46).  In  order  that 
(45’)  reduce  to  (45)  when^  and^  are  independent,  we  require 
ui  ~  »  ^i  >  Aj^)  so  chosen  that 


(47) 


u(Yu.  Y2i »  °)  =  0. 


We  now  investigate  further  properties  of  the  p  (i).  We  require 

r  s 

the  following  five  conditions  concerning  p  ( i) 

rs  '  ' 

(i)  The  probability  associated  with  any  one  cell  is  greater  than 
zero  and  less  than  one.  Hence,  for  pu(i)  we  have 


0  < 


E.  ♦>)(.♦>)♦  .j -1 « 


(11)  Hie  probability,  pl#(l),  cap  be  written  explicitly  by 
using  the  equality  pu<l)  +  ,^(1)  .  (I)  ,  uhlch 

from  Table  2.  Hence, 


comes 


(48) 
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<«-(lt.Y“)'l.[(1+.Yll)(lt.^+u1]'1. 


Since  p10(i)  has  the  same  restriction  as  Pll(i)f  we  have 

#<(,+  'V-[(**e',lX>*eY2l)*.Jl<l. 


The  manner  in  which  the  u(vH,  Y 

no  means  unique.  The  constant  covariance 

„  iailLe  between  me  two  charac- 

terietice  seems  appropriate  for  the  present  situation,  however  and  is 

J"  -  o(vCh  Slvmp  er  th3n  SOme  more  8eneral  function  such  as  ’ 


Al2)  is  incorporated  is  by 
term  between  the  two  charac- 


1 2  i 


li’ 


y21)  • 


O 
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( iii)  Similarly,  since  Pu(i)  +  PQ1(i)  =  P  ^i), 

p0i<i)  -  (l  +  eY21)  1  -  [(l  +  eVli)(l  +  eY21)  +  uj  ‘  . 


Again,  0  <  pQ1(i)  <  A  so  that 


v2i\_  1  17,  .  _Yu\A  ,  y2i 


0  <  (l  +  e'21)  -[( 


1  +  e  “  1  +  e 


)  *  -  J 


(iv)  Now,  pQ0(i)  =  1  -  p1Q(i)  -  pQ1(i)  -  Pu(i),  with  the  usual 
restriction  0  <  Pqq(1)  <  1*  Thus, 

,  A  Yli\_1  A  .  y2l\l  .  |Y,  .  Yli V,  .  Y2i 


(50)  P00(D  -  1  -  (l  +  «  “)  -  ('  +  '  21)  +  [( 


1  +  e  1  +  e 


)  ♦  “  J 


0  <  1  -  (l  +  eYliy‘  -  (l  +  e"21)’1  +  [(l  +  a’11)^  +  e’21)  +  uj  '  <  1. 


(v)  Let  he  the  correlation  coefficient  between  the  two 
characteristics  and^.  Then  noting  that 

L  cov(^,^]  given  i) 

P,2  -  ,  » 

Jvarty^  I  given  i)  var(7^2  j  given  i) 


(51)  p, 


we  have 

2  J  Yli  +  Y2i  A.  y2i\'^ 


Ku)  K2i) 


Since  p^2  is  a  correlation  coefficient  it  must  lie  between 


- 1  and  1 .  Thus  , 

rK1^721)-, 

J  Vu+Vji  /  Yu> 


'l+eYuVl«Y2i' 


<  1  . 


K“)  K2i) 
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We  can  consolidate  conditions  (i),  (ii),  and  (iii)  into  the  following 
theorem. 


Theorem_J_.  Conditions  ( i)  ,  (ii),  and  (iii)  hold  if  and  only  if 


(52) 


.Yki  .  Yli+Y2i 
e 


)]• 


Proof. 


>  k™:2  [-(« ki  - 
<a)  Sssu“e  ui  >  k!!*2  [-(eYkl  +  «Vu+Y21)j 


Then,  u^ 

.  (y li 

>  -(e 

Y2i 

+  e  + 

Yli+Y2i\ 
e  ) 

,  which  implies 

+  u  >  1. 

Hence , 

condition 

( i)  follows. 

Furthermore , 

max  (  \i\ 

k-1,2  (1+e  ) 

<  (1+eYll)(l+eY2i; 

)+  V 

and  since 

(-Y2i) 

+  u.  >  0, 

we  have 

max  / 
k-1,2  \ 

.+<>)  ‘ 

<>♦[( 

'l«Yll)(l+eY21)+  u,]  ‘ 

vu 


Thus,  conditions  (ii)  and  (iii)  follow. 

(b)  Assume  conditions  ( i)  ,  (ii),  and  (iii)  hold.  Then  from 
conditions  (ii)  and  (iii)  it  follows  that 


[-(> ♦  <Y‘1+Y21) 


] . 


Q.E.D. 


Theorem  2.  It  is  necessary  and  sufficient  that 

Yu)(^Yz1' 

Yli+Y2i' 


(53)  “2  +  =YU+Y2t)j  <  «.  <  (‘+e  “X*8  “)  C<YU+Y21> 

1  (l-eYll+Y21) 
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(54) 


in  order  that  conditions  (i)-(iv)  hold,  provided  y^  +  V2^  <  0* 

Otherwise,  (52)  provides  the  necessary  and  sufficient  condition. 

Proof,  (a)  Assume  y  +  y^^  <  0,  and  that  (53)  holds.  Then 

u^  >  '  "(e  +  e  ^  ^mP^es  conditions  (i)-(iii)  hold. 

Further,  this  implies 

[l+eVliJ1+[l+eY2‘J1>[(l+eYli)(l+eV2i)+Ui]'1, 

since 

k-  [1«Y-]1>[(l+eY“)(1«Y-)+Uif. 

Finally ,  u  <  (.^  -)  tapH„  that 

‘  (l-aYll+Y21) 

1  +  [(l+eYll)(l+eV21)  +  uj'  >  (l +eYu)'‘  +  (U-*11)1  , 

for  the  case  y^  +  <  0. 

(b)  Assume  y  +  y2^  <  0,  and  that  conditions  (i)-(iv)  hold. 

Then  a  reversal  of  the  argument  in  (a)  establishes  (53) . 

(c)  If  y  +  y^ ^  2  0,  it  is  then  easy  to  show  by  an  argument 
analogous  to  the  proof  of  Theorem  1  that  (52)  provides  a 
necessary  and  sufficient  condition  for  conditions  (i)-(iv) 
to  hold. 

Theorem  3.  (a)  Assume  y  +  y^^  <  0.  Then  condition  (v)  holds 


if  and  only  if 


I 


(55)  (l+eYliJ(i+eY2i) 


All  +  V2tv  2  “l 


n  a  :  121  n 

1  -  el  2  ^ 


^Yli  +  Y2i’ 


'“X-21) 


■ ♦ x-^4^) 

4 


(b)  Assume  Y[i  +  y2i  *  0.  Then  condition  (v)  holds  if 
if 

/Yli  +  V2i \  Y 

(  i+PYliv,  Y2n 


and  only 


33^) 

(hi  1  hi\ 

1  +  e^  2  / 


Proof.  Rewriting  condition  (v)  we  see  that 
/Yli+Y21N 

6  2  ;[(^VliX^Y2I)]'1S[(l+e¥“X.+eY21)+u1J1 


Vll)(l+eV2‘)]-1  «  .(■ 


~Y  1  i"*"  y2  i  ’ 


V  o  ••  \  -T  ^ 


T(1+e  Xi+e  2i)] 


A  manipulation  of  this  inequality  for  the  two  cases  ,  y  +  y  <  0 

and  Yu  +  Y21  2  o  establishes  (54)  and  (55)  respectively. 

Theorem  4.  Suppose  §  <  0.  Then, 


(i-*V2)>  i-.s  • 

Proof.  (56)  follows  because  e^2  -  e3/2§  >  e5  _  3 /2 § 


K 


f 


Theorem  5.  If  y2^  >  Y^>  then 

All  v21) .  VUw  Y2is 

Vli  /.  V2n  7  2  '(1+e  Xltc  )  V21  ,,  vm 

(57a)  e  (1+e  ) <  - — v11'^71^ - * e  (1+e  )  ■ 


Otherwise,  if  y^  2  ^2i' 


Ah  V2l\,  vuw  Y2n 

<57b,  (l+eVli)  ,  e  ^  /(1-t~e  X^_;  ,  .hi  (1+eV21) 


rroot-  If  y21  >  Yu > 


e  1+e 


r  /Y,,+Yo,y1  ^ Yli+Y2i ^ 

V - 2 - 1  t  Yli\ 

1^1  2  ;  <  e  (l+e  U) 


The  other  inequalities  are  proved  in  a  similar  manner. 
Theorem  6.  (a)  Assume  y^  +  y  <  0  and  y^  >  Y^.  Then 

conditions  (i)-(v)  hold  provided  that 


(58a)  -eVl1  (l+eV21)  <  u,  < 


l+aYl‘)(n-eY2t)  (Yll+Y2l) 


Yli+Y2i' 


If  y  2  y 2 i ,  then  conditions  (i)-(v)  hold  provided  that 


hi  A  V„v  (l+eVu)(1+eV21)  (Yli+Y2i)  . 

-e  1+e  I  <  u  <  - ; -  e 

V  '  (l-eYli  Y2i) 


(58b) 


(b)  Assume  y^  +  y2  s  0.  If  y2i  >  Y^,  conditions  (i)-(v)  hold 


provided  that 
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(59a) 


-e 


(*«V2i)  <  Y 


If  y^  £  ^21’  t*ien  condltions  (i)-(v)  hold  provided  that 


(59b) 


-e 


Y2i  *11 


(l+e  U)  <  u.  . 


Proof,  (a)  Theorem  2  provides  the  upper  inequality  for  u,  in 
order  that  conditions  (i)-(iv)  hold.  Theorems  3  and  A  show  that 
no  adjustment  is  required  when  condition  (v)  is  added.  The  lower 
inequality  for  u.  follows  from  Theorems  2,  3,  and  5,  depending  on 
whether  .  >  y,  .  or  y„ .  y, . . 

(b)  If  y^.,  +  Y2^  2  0,  there  are  no  upper  bound  requirem.nts  on 
u^  with  respect  to  conditions  (i)-(v). 


We  now  consider  an  example  incorporating  specific  assumptions 
about  the  form  of  u.  Let 

Yli+Y2t  .  A12 
=  e  (e  -  1). 

Then 

YU  Y2l  "Y,  i+V2  i+Ai  o 

(60)  pu(i)  «  l  1  +  e  +  e  Zi  +  e  U  2l  12y  . 


max 


1  /  Yki  ^li+Y2i\- 


Clearly,  u^  >  ^  "  +  e  ”  ~~J  »  so  that  the  lower  inequalities 

in  Theorem  6  are  satisfied.  We  need  only  be  concerned  with  the  upper 
inequalities  if  y  +  Y2^  <  0.  In  this  case,  we  must  have 

Y,,w  Y- 


(61) 


e  -  1  < 


l+e 


u)K2t) 

(l-eY‘1+V21) 


satisfied  for  all  yu  and  y^  ^  such  that  y^  ^  +  y.,  .  <  0.  Thus,  -®  <  & 
<  In  2  assures  that  conditions  (i)-(v)  are  me l . 
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We  now  sketch  the  development  of  the  maximum  likelihood  equation. 


Suppose  we  make  observations  on  N  individuals ,  each  of  which  can  be 
measured  on  a  zero-one  basis  with  regard  to  the  two  characteristics 
7^  and  7^2*  Let  y 1 )  be  the  random  variable  that  takes  the  value  1 
if  the  outcome  on  the  i-th  individual  is  (j  ,k)  with  respect  to  and 
7|2»  respectively,  1  =  1,  ...,  N;  J,  k  =  0,  1.  Otherwise,  y ^ ^ ( i )  takes 
the  value  zero. 


(62) 


pr  fyJk(i)  ■  0  ■  Pjk(o. 


Thus,  the  likelihood  L  of  a  sample  of  size  N  is  equal  to 


(63) 


n 

j,k=0,l 


N 

n 

i-l 


[PjkU)l 


V1* 


where 


(64) 


S  PjJ1)  =  S  yik(i)  " 

J.k  J  j,k  Jk 


Letting  L  represent  the  likelihood  and  X  “  In  L,  we  have 
N 

(65)  X  -  Xj  [y^U)  lnepn( l)  +  yio(i)  lnep10(i) 

+  y01U)  ^gPoi^1)  +  y00(i)  lneP00^1^ 

Now,  Eq.  (60)  determines  Pjk(l)  as  a  function  of  £  and  A^*  3’hus , 
with  some  algebraic  simplification 
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(66) 


*  a  -  S  lne  (l  +  eYli  +  eY2i  +  eYli+Y2i+A12^| 


)  -  in  ( 

1  +  v“Yi 

'  e  \ 

/  J 

+  E  y10<t)[v21  +  lne  (l  teYli+412 


+  £  W’u  +  ‘"e  (‘  +  eV2I+il2)  -  lne  (l  +  e"21)- 


N 

+  W  y0O(1)[(vlt  +  V2i)  +  lne  (2  +  eYu  +  eV2i  .  e^12^  .  eVli+T 


21 


lne  (l  +  eV“)  .  U  (l  ♦  >)]. 


Recalling  that  .  [„0  +  jj'x^  and  y^ 


LB 


lions  „hose  solntlon  yields  the  «*!„„  likelihood  estimate 
by 


the  equa¬ 
te  given 


(67) 

O 

M 

(68) 

H-  -  0  , 

*1 

1  =  1 »  •  • • ,  s- 1 

(69) 

II 

O 

(70) 

O 

II 

and 

J  ^  >  •••>  t*  1 

here,  for  example,  (67)  is 


Thus,  the  fitted  p  (i)  values  denoted  by  p,R(i)  are  8iven  by 


A.  _ _ ^ 

(73)  $  Ai)  -  VU  V2i  *li^2i+*12’ 
11  1+e+e  +e 


( 7 A)  PQ1(i) 


W.  ,  ^2i+^12N 

e  \l+e _ i 


(,  +  eV“  +  .*«  +  ♦  e  2 “) 


.M,  „  .hi  *12) 

(75)5,0(1)'--,  ,n 

(1  +  e  +o  +  c  Jt, 


1  +  e 


(76)  Pnnl1)  “  C.  .\  .  Yu  Vn+^4+^12 
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V.  USE  OF  THE  MODEL:  AN  EXAMPLE 


Logit  models  provide  excellent  opportunities  for  specific  types 
of  decisionmaking.  If  a  choice  must  be  made  between  different  courses 
of  action,  then  the  decisionmaker  can  consider  the  various  probabilities 
of  success  as  determined  by  the  specified  model.  The  parameters  of 
this  model  would  be  estimated  from  past  performance,  using  the  techniques 
developed  in  the  previous  sections.  Similarly,  if  the  decision  does 
not  involve  choosing  between  alternatives,  but  rather  involves  deciding 
whether  to  "choose  one  alternative"  or  not,  the  decisionmaker  can  con¬ 
sider  the  probability  of  success  of  this  one  course  of  action  as  de¬ 
termined  by  the  posited  logit  model  and  then  base  his  decision  on  some 
a  priori  probability  level. 

Frequently,  the  determinations  of  the  probabilities  of  success  of 
two  different  classes  of  alternatives  are  not  independent.  Hence, 
models  that  consider  such  possible  dependence  will  be  of  more  value  to 
the  decisionmaker.  The  dependent  logit  models  developed  above  are  such 
models . 

This  section  presents  an  example  that  describes  the  determination 
of  the  logit  model  parameters  under  three  different  assumptions  that 
will  be  explained.  The  data  for  the  example  are  data  on  Category  IV 
airmen  who  were  admitted  to  the  service  in  1960  under  a  special  program. 
These  data  were  collected  into  a  specific  file  known  as  the  "Dual  2b" 
data  file. 

To  be  admitted  to  this  special  program,  a  potential  recruit  had 
to  achieve  a  score  of  25  (out  of  100)  on  at  least  two  AQEs.  However, 
a  score  of  40  (out  of  100)  on  any  one  AQE  would  qualify  him  for  "regular 
enlistment,"  and  hence  would  preclude  participation  in  the  "Dual  25" 
program. 
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The  basic  hypothesis  is  that  the  probability  of  reenlistment 
after  an  initial  4-year  term  by  Category  IV  airmen  is  related  to  their 
initial  AFQT  score,  their  race,  and  their  age  at  enlistment  by  means 
of  the  logistic  function.  Thus  the  logit  relation  can  be  written 

(77)  logit  (r)  =  in  (-y—)  =  Oq  +  +  <*2X2i  +  ^Si’ 

where  m  the  AFQT  score,  X2  =  a  dummy  variable  for  race  (0  for  Negro, 
1  for  Caucasian),  and  =  an  age  scale  variable. 

If  we  assume  that  the  reenlistment  rate  of  all  enlistees  is 
determined  by  the  independent  variables  X^  ,  X^  ,  and  in  the  same  way, 
then  the  parameters  of  (77)  can  be  estimated  straightforwardly  by  using 
"simple  least  squares"  regression  analysis.  The  results  are 

(78)  logit  (r)  =  9.532  -  0.326X1  -  1.696X2  -  0.215X3> 

(2. 125) (-1. 907)  (-2.244)  (-1.526) 

R2  =  0.  1757  , 

(SE)2  =  0.7509, 

where  the  numbers  in  parentheses  are  the  "t"  values  associated  with  the 
parameter  estimates,  and 


Mean 

Va  r iance 

Logit  (r) 

-0.4711 

0.8243 

X1 

25.1310 

0.6179 

X2 

0.5548 

0.0596 

X, 

4.0095 

1.7204 
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Now  assume  that  the  enlistees  can  be  categorized  into  two  groups: 
those  with  a  high  school  education  and  those  without.  Separating  the 
sample  into  its  appropriate  parts,  we  can  estimate  (77)  ior  each  group. 
Thus 

(1)  high  school  education  completed: 

(79)  logit  (r)  =  13.420  -  0.594X1  -  1.630X2  +  0.313X3> 

(2 . 67 1 ) ( -2. 926)  (-1.385)  (1.423) 

R2  =  0.3766, 

(SE)2  =  0.5242, 

where  the  numbers  in  parentheses  are  the  "t"  values  and 

Mean  Variance 
X  24.971  0.6687 

X2  0.3762  0.0247 

X3  5.1143  0.7460 

Logit  (r)  -0.4238  0.8409 

(2)  high  school  education  no t  completed: 

(80)  logit  (r)  =  8.019  -  0.216X1  -  1.124X  -  0.762X3? 

(1.5  05) ( -1 . 068)  (-1.309)  (-2.625) 

R2  =  0.3569, 

(SE)2  =  0.4147, 

where  the  "t"  values  are  given  in  the  parentheses  and 


Mean 

Variance 

logit  (r) 

-0.4714 

0.6449 

X1 

25.2900 

0.5161 

x2 

0.7333 

0.0308 

X5 

2.9048 

0.2538 
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Assume  now,  however,  that  the  two  groups  do  not  function  independently. 
We  then  estimate  parameters  of  each  equation,  given  this  dependency 
assumption.  The  residual  covariance  matrix  between  Eqs.  (79)  and  (80) 
is  given  by 

(0.5242  0. 1463\ 

0.  1463  0.4147/ 

A 

The  inverse  of  E  is  then  used  as  developed  in  Secs.  II  and  III  to 
u 

obtain  "better"^  estimates  for  the  parameters  of  (79)  and  (80).  Thus 
we  obtain  the  following  estimate: 

(1)  high  school  education  completed: 

(81)  logit  (r)  =  10.330  -  0.487XL  -  1.338X2  +  0.373X3 

(2. 149) (-2. 497)  (-1.195)  (1.767) 

and 

(2)  high  school  education  no t  completed: 

(82)  logit  (r)  =  5.446  -  0.114X  -  0.757X2  -  0.850X3, 

(1.074) (-0.595)  (-0.927)  (-3.035) 

where  the  "t"  values  are  given  in  parentheses. 

Thus  depending  on  the  stringency  of  the  assumptions  the  decision¬ 
maker  is  willing  to  endure,  the  best  estimates  for  those  assumptions 
may  be  obtained  as  above. 

One  other  case  may  be  illustrated  by  using  this  example.  Assume 
that  the  group  of  high  school  graduates  is  different  from,  but  not  in¬ 
dependent  of,  the  group  of  non-high  school  graduates.  Further,  assume 

The  estimates  are  "better"  in  the  sense  manifested  in  Secs.  II 
and  III,  specifically  Eq.  (37). 
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that  the  difference  is  manifested  merely  by  some  constant  amount, 
whereas  the  effect  of  each  independent  variable  on  the  probability 
of  success  is  the  same  in  both  groups.  In  this  case  we  "constrain" 
the  parameter  values  ,  ot^  >  a nd  a ^  of  (77)  to  be  the  6ame  for  the 
estimates  of  (81)  and  (82).  Thus  we  obtain 

(83)  logit  (r)  =  {iq*81o}  '  °-383Xi  '  2-085x2  ‘  O*02^, 

where  the  intercept  value  10.050  is  associated  with  high  school  gradu¬ 
ates  and  10.810  is  the  intercept  for  non-high  school  graduates. 

We  have  used  this  set  of  Category  IV  airmen  data  to  illustrate 
how  the  techniques  developed  in  Secs.  Ill  and  IV  can  be  implemented; 
and  although  the  various  assumptions  employed  are  not  necessarily 
tenable,  their  use  should  suffice  to  demonstrate  the  possible  im¬ 
plementations  of  the  "dependent  logit  models." 
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APPENDIX 

2 

The  derivations  of  the  maximum  likelihood,  minimum  x»  and  minimum 
2 

logit  x  estimates  are,  for  the  most  part,  well  known.  Hence  the 
various  methods  will  merely  be  sketched  as  an  aid  to  those  unfamiliar 
with  that  body  of  literature. 


1.  MAXIMUM  LIKELIHOOD 

Let  P  represent  the  probability  that  an  event  E  will  occur,  and 
further> let  this  probability  depend  on  an  exogenous  variable  X  by 
means  of  the  logistic  function 


(84) 


P 


1 

!  +  e-(<*+6X)  * 


Construct  the  following  likelihood  function  from  a  sample  of  size  n: 
(85) 


n  y t  1-y. 

L  -  n  P  1  (1  -  p.)  \ 

i-1 


where  y^  ■  0  or  1  for  each  i. 
Then 


n  n 

y. 


(86)  I  ■  In  L  *  2J  1°  +  2J  In  (1  -  p  )  ^ 

i-1  i-1  1 


S  U  '  yi>  ln(i  1  ,-<<*«>)  '  S  y‘  ln  (l  +  e'(v4*X)). 


and 


91  T'  /  -  % 

s  ’  ft  -  fi>  ■  °. 


(87) 


[>I»] 


r  -(a46Xi)  - 1 

>re  pi  =  yi/ni  and  =  j^l  +  e 


2.  MINIMUM  x  • 

Let  P,  the  probability  that  an  event  will  occur,  be  the  same  as 

in  (84)  above.  Consider  a  sample  of  from  the  population  and  observe 

the  successes,  say  y  ,  in  a  success-fail  mode.  The  observed  probability 

of  a  success  is  then  y^/k^,  and  the  probability  of  a  failure,  1  -  y^/k^. 

The  expected  probability  of  success  is  p^ ,  where  p^  is  defined  as 

above;  of  failure,  1  -  p^ . 

2 

The  \  statistic  can  then  be  written 

2  v- /N(>'i/ki  -  fi>2  ,  (fi  ‘ 

(89)  x  - FI - +  — r-p—  r 


(89) 

n 

2  V- 

X  = 

i  =  l 

and 

2 

(90) 

X  = 

2  V1  N  ,  *  .2 

X  ~  w  V1  '  Pi)  yi7  i  "  Pi  * 


Let  =  y^/k^  and  be  the  estimate  of  P^  ,  the  true  probability 
2 

of  success.  Then  the  x  statistic  is  minimized  when  the  following 
equations  are  solved: 


o 


fpi( i  -  pt)  +  (i  -  pt)Pi" 


Pla  -  pp 


(Pt  "  Pt)  "  °. 


Cpj(  1  -  p.)  +  ( 1  -  Pi)pJ 


Pi  -  Pi 
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3.  MINIMUM  LOGIT  x 

Consider  samples  out  of  the  population,  i  =  1,  . . jj.  Let 
the  observed  proportion  of  successes  p.  be 


(93) 


pi  =  Pi  +  V 


where  P.  is  the  "true  proportion  for  the  ith  sample,  and  u.  is  a  bi- 
nomially  distributed  random  variable  such  that 


and 


E(ui)  =  0, 

Pi(l  -  Pi> 
Var(u  .  )  =  — - L_ 

1  ni 


E(ui.u j)  =  0  for  i  +  j- 


We  wish  to  minimize,  then, 

n 


E  wi(pi  -  pi>  > 

i  =1 

-here  u.  I,  Inversely  proportioned  to  the  variance  of  p,  ,  and  p;  U  the 

estimate  of  P..  Observe  that  for  a  rather  small  difference,  (p  .  S  )2 

i  pi 

can  be  approximated  as  follows; 

(94>  (Pi  '  Pi)2  ~  1  '  P^lfPrd  -  P.)V.  -  l  )2 

1  1  i  l  i  ’ 

where  lL  =  l„(p,/(l  -  p.)),nnd  l  =  „  +  gx 

Consider,  then,  (90)  above: 


(95)  x2  =  E 


ft  iyT-nr7(pi 


pi> 


ZVl 

i  =  l  1  1 


d  -  Pi)(?i 


V 


-46- 


Since  all  the  terms  in  the  approximation  are  linear,  ordinarily 
least -square  regressions  with  weights  w^  [ p^ ( 1  -  p  ) ]^  can  be  used  to 
obtain  estimates  of  <7  and  8,  and  subsequently,  p^  for  all  i.  Thus 

.  2 

<«>  g-  •£»,(>  •  •  \)  •  °. 

a  2 

(97)  af~  =Snixi(1  '  pi>pi(2i  '  V  "  °- 

Equations  (96)  and  (97)  are  the  normal  equations  for  ordinary  linear 
regress  ion . 


-47- 


REFERENCES 


1.  Aitken,  A.  C.,  "On  Least -squares  and  Linear  Combination  of  Observa¬ 

tions,"  Proceedings  of  the  Royal  Society  of  Edinburgh,  55 ,  1934-35, 
pp.  42-48. 

2.  Berkson,  J.,  "A  Statistically  Precise  and  Relatively  Simple  Method 

of  Estimating  the  Bioassay  with  Quantal  Response,  Based  on  the 
Logistic  Function,"  Journal  of  the  American  Statistical  Associ- 
atlon,  48,  1953,  pp.  565-599. 

2 

3.  - ,  "Minimum  \  and  Maximum  Likelihood  Solution  in  Terms  of  a 

Linear  Transform,  with  Particular  Reference  to  Bioassay,"  Journa 1 
of  the  American  Statistical  Associat ion  ,  44 ,  1949,  pp.  273-278. 

$.  - ,  "Approximation  of  Chi-square  by  'Probits 1  and  ’Logits'," 

Journal  of  the  American  Statistical  Association,  41 ,  1946, 
pp.  70-74. 

5.  - ,  "Application  of  the  Logistic  Function  to  Bioassay,"  Journa 1 

of  the  American  Statistical  Association,  39 ,  1944,  pp.  357-365  . 

2 

6.  - ,  "Maximum  Likelihood  and  Minimum  \  Estimates  of  the  Logistic 

Function,"  Journal  of  the  American  Statistical  Association,  50,  1955, 
pp.  130-162. 

7.  - ,  "Tables  for  che  Maximum  Likelihood  Estimates  of  the  Logistic 

Function,"  Biometrics  ,  13  ,  1957  ,  pp.  28-34. 

8.  Cox,  D.  R. ,  "The  Regression  Analysis  of  Binary  Sequences,"  Journa 1 

of  the  Royal  Statistical  Society,  Series  B,  2_0,  1958,  pp.  215-242. 

9.  Donaldson,  T.  S.  ,  and  A.  F.  Sweetland,  Trends  in  F-101  Aircraft 

Maintenance  Requirements,  The  RAND  Corporation  ,  RM-4930-PR, 

April  1966. 

10.  Dyke,  G.  V.  ,  and  H.  D.  Patterson,  "Analysis  of  Factorial  Arrange¬ 
ments  when  the  Data  are  Proportions,"  Biometrics  ,  8  ,  1952,  pp.  1-12, 

,  11.  McCall,  J.  J,,  and  N.  Wallace,  Training  and  Retention  of  Air  Force  ^  y  >1 

Airmen:  An  Economic  Analysis,  The  RAND  Corporation,  RM-5384-PR  n  f 1  ‘ 
August  1967.  ’  ikX> 

12.  McGlothlin,  W.  H. ,  and  T.  S.  Donaldson,  Trends  in  Aircraft  Maintenance 

Requirements ,  The  RAND  Corporation,  RM-4049-PR,  June  1964. 

13.  Walker,  S.  H.  ,  and  D.  B.  Duncan,  "Estimation  of  the  Probability  of 

an  Event  as  a  Function  of  Several  Independent  Variables,"  Biometrics, 

1967,  pp.  167-180. 


-48- 


14.  Zellner  ,  A.,  "An  Efficient  Method  of  Estimating  Seemingly  Unrelated 

Regressions  and  Tests  for  Aggregation  Bias,"  Journal  ot  the 
American  Statistical  Association,  June  1962,  pp.  348-368. 

15.  - ,  and  T.  H.  Lee,  "Joint  Estimation  of  Relationships  Involving 

Discrete  Random  Variables,"  Econometrics ,  April  1965,  pp.  382-394. 


DOCUMENT  CONTROL  DATA 

fl  ORIGINATING  ACTIVITY 


2o  REPORT  SECURITY  CLASSIFICATION 
UNCLASSIFIED 


THE  RAND  CORPORATION 


2b.  GROUP 


3.  REPORT  TITLE 

ESTIMATION  TECHNIOUES  FOR  DEPENDENT  LOGIT  MODELS 


4.  AUTHOR(S)  (Lott  nom«,  first  name, Initial) 

Cook,  Jr.,  A.  A.  and  A.  J.  Gross 


5.  REPORT  DATE 

November  1968 


7  CONTRACT  OR  GRANT  No. 
F446 20-67 -C-0045 


9o.  AVAILABILITY/ LIMITATION  NOTICES 
DDC-1 


6a.  TOTAL  No.  OF  PAGES 

59 


6.  ORIGINATOR'S  REPORT  No. 

RM-57  34 -PR 


6b.  No.  OF  REFS. 


10.  ABSTRACT 


A  discussion  of  the  estimation  techniques 
of  logit  models  whose  response  variables 
are  noc  necessarily  statistically  inde¬ 
pendent.  The  techniques  developed  and 
used  include  the  Generalized  Least  Squares 
approach  of  Zeliner;  stepwise  regression 
and  its  relevance  in  estimating  logit  model 
parameters;  and  the  "contingency  table" 
concept  of  fixed  marginal  probabilities, 
which  is  used  to  develop  e  bivariate  logit 
model  that  possesses  certain  statistical 
and  probabilistic  properties  concerning 
the  correlation  coefficient  and  the  sum¬ 
mation  of  individual  cell  probabilities  to 
unity.  The  techniques  are  designed  for 
application  to  two  Air  Force  problems: 

The  first  problem  uses  concomitant  informa¬ 
tion  to  estimate  the  probability  that 
first-term  airmen  reenlist;  the  second, 
information  concerning  flying  missions  to 
estimate  the  probabilities  that  inter¬ 
dependent  aircraft  subsystems  fail  during 
these  missions.  Some  applications  to  the 
problems  are  presented,  but  this  study  is 
mainly  directed  toward  development  of  the 
necessary  techniques.  An  example  illus¬ 
trating  the  various  techniques  is  given. 


9b.  SPONSORING  AGENCY 

United  States  Air  Force 
Project  RAND 


II.  KEY  WORDS 

Reliability 

Maintenance 

Statistical  methods  and  processes 

Air  Force 

Personnel 

Models 


